The present invention relates to an apparatus and method for performing image sharpening in confocal microscopy, ofxe2x80x94in particularxe2x80x94the Z axis of confocal data sets in real time with a single scan. The present invention is of particular but by no means exclusive application in increasing the density of information storage of optical date storage devices, particularly of three dimensional digital data store devices.
In confocal microscopy it is generally desirable to minimise the thickness of the focal plane section. This is achieved by reducing the return pinhole to the smallest size which will give a reasonable signal.
With a 1.4 NA oil immersion objective lens, the XY resolution is approximately 200 xcexcm while the Z resolution is approximately 500 xcexcm. This means that the voxels or boxels making up the image have a long axis that is 2.5 times the two orthogonal voxel dimensions. This is true for all laser scanning confocal microscopes (LSCMs) and affects all 3D reconstructions.
This ratio is greater for lower NA lenses and the result has a deleterious effect in 3D reconstructions. Rotations of images show a lack of resolution in the Z direction and perhaps more seriously, an artefact in which give a perception an anisotropy in views of tissues which include a Z dimension.
Image processing software can be used to improve the image. For example, in a first existing technique, Z sharpness is increased by concentrating on a voxel and then deconvolving it to a sharper value by subtracting from it a small proportion of the value of the voxel above it and below it.
A second existing technique utilises a similar principle in conjunction with XY sharpening algorithms. This is actually marketed as a synthetic aperture confocal system which can deconvolve sharp pictures from successive depth blurred low contrast brightfield images. However, it has been suggested that confocal data sets would also benefit from this approach. More sophisticated correction takes into account the brightness of pixels two levels above and below the focal plane being Z sharpened.
These are in effect a digital versions of unsharp masking techniques by means of which a correction is provided for the brightness of each individual voxel, which takes into account the brightness and xe2x80x98spilloverxe2x80x99 addition of light from voxels above and below. The successful use of the aforementioned methods also depends on the operator having a fairly good understanding of the nature of the sample, the lens characteristics, the pixel sampling interval, the distance between successive image planes and other factors and entering these into the variables and chart of the algorithm.
A third existing technique that effectively achieves an identical Z sharpening result involves carrying out two separate scans of each plane, one scan being with the pinhole stopped right down and a second scan with the pinhole opened to about double the XY resolution optimum size. The second scan includes light from fluorescence from objects in the adjacent planes above and below and gives an analog sum of light intensities which can be used to obtain a correction factor equivalent to the digital correction algorithm used in the technique described above (in which one concentrates on a voxel and then deconvolves it to a sharper value by subtracting from it a small proportion of the value of the voxel above it and below it).
However, the above methods are time consuming and require a knowledge of the lens characteristics and sampling intervals. They require more than one scan to be made together with post acquisition processing. The software deconvolution (which is effectively digital unsharp masking) requires 3 or 5 scan depths to obtain corrections for 1 and 2 planes above and below the plane to be sharpened and, in some techniques, 2 or 3 scans with 2 or 3 different pinhole sizes.
Similarly, many methods have been proposed for high density digital storage using optically addressable elements within the three dimensional structure. Typical of these is the work by Rentzepis and by Min Gu. Previously proposed methods use confocal techniques to address the individual bit storage elements. The resolution in XY and Z of these methods has pretty thoroughly been established by Sheppard, Gu and others.
FIG. 1 illustrates the formation of a Gaussian Waist 10 when a TEM00 beam 12 comprising a set of plane parallel wavefronts 13 from a laser 14 passes through a beamsplitter 16 (in which the first reflection is omitted for clarity) and objective lens 18. The lens 18 produces a convergent concentric wave front 20. If the Gaussian Waist 10 is focussed in a uniform fluorescent medium (not shown) then the points of re-emission of light which will return more than a given percentage of the excitation light energy through the return pinhole 22, after reflection and re-direction by beamsplitter 16 and focussing by lens 23, will constitute a volume 24 which is roughly football or elliptically shaped, symmetrically located in the waist 10. This elliptical volume 24 could be termed an isofluorescence boundary for confocal pinhole return. In fact for a perfect lens the xe2x80x98footballxe2x80x99 has two haloes above and below it (not shown). These do not affect the discussion and have been omitted for clarity. The 1/e2 Gaussian profile is also indicated in this figure, as is the region 28 shown in subsequent figures and encompassing the Gaussian Waist 10 and environs.
Clearly the principle of unsharp masking involves the subtraction of return light from just above and just below the pixel to be sharpened in which the xe2x80x98overlapxe2x80x99 return light is taken away from the central pixel.
Two such prior art techniques (such as those employed in the first and second existing techniques discussed above respectively) are illustrated in FIGS. 2A and 2B, in which all the boxels are the same size. The pinhole is not altered but the xe2x80x98overlapxe2x80x99 required for the unsharp masking is obtained from the pixels in the scans on either side. FIG. 2A illustrates a prior art digital image sharpening technique using three scans at three separate levels within a specimen. In FIG. 2A, the plane to be sharpened is indicated at 30, and cross sections of the Gaussian Waist and confocal volume (or isofluorescence intensity voxel perimeter) for each of three scans are shown at 32, 34 and 36; the Gaussian Waist and confocal volume are respectively on, above and below the desired focal plane. In the sharpening procedure (see schematic representation at 38), a portion of both dotted volumes 40 and 42 (corresponding to the confocal volumes of the second and third scans 34 and 36) are removed from the central volume 44 (corresponding to the confocal volume of the first scan 32), leaving a sharpened voxel 46.
In the prior art technique illustrated in FIG. 2B, the central voxel 50 is sharpened by removing a portion of a 3xc3x973 voxel matrix 52 from above and another 3xc3x973 voxel matrix 54 from below the desired focal plane. The schematic image of FIG. 2B is shown undersampled from the Nyquist point of view to increase clarity.
FIG. 3 illustrates the traditional unsharp masking of the third existing technique discussed above, in whichxe2x80x94after a first scan 60 is made with the pinhole stopped downxe2x80x94a second scan 62 is made with the pinhole opened but at the same focal plane. Next the pixel values for the image produced in the second scan 62 are subtractedxe2x80x94where an overlap existsxe2x80x94from the image produced in the first scan 60 (with the pinhole stopped down); the resulting difference signal contains the xe2x80x98overlapxe2x80x99 information 64 and is used to correct each of the pixels to be sharpened to produce the sharpened voxel 66.
It is an object of the present invention, therefore, to provide a method that avoids the necessity for multiple scanning and post acquisition processing.
It is another object of the present invention to provide a method and apparatus for reducing the Bit Error Rate (BER) of reading and of increasing the storage capacity (typically measured in gigabits per cubic millimeter) of a data storage material.
In a first broad aspect, therefore, the present invention provides a method of image sharpening in a confocal microscopy or endoscopy observation, comprising:
collecting true confocal return light emanating from an observational field of an object;
focussing said true confocal return light into a core of a fiber wave-guide;
collecting near confocal return light from a volume partially overlapping said observational field and thereby defining an overlap volume;
focussing said near confocal return light into said fiber wave-guide so as to be transmitted principally in a cladding of said fiber wave-guide;
separately detecting said true confocal return light and said near confocal return light to produce a true confocal output signal and a near confocal output signal; and
adjusting said true confocal output signal on the basis of said near confocal output signal to substantially eliminate from said true confocal output signal a component due to said near confocal output signal;
whereby the effective volume of said observational field is reduced and the resolution of said observation is effectively increased.
Preferably said overlap volume is in the Z axis of said observational field.
Preferably said true confocal return light and said near confocal return light are collected and focussed by means of a light condenser. More preferably said light condenser comprises a lens or a compound lens.
In one embodiment, adjusting said true confocal output signal comprises subtracting said near confocal output signal from said true confocal output signal.
This may constitute an over-correction, but the component of the near confocal signal due to light from other than the overlap volume will be small compared to the component of the near confocal signal due to light from the overlap volume, so the adjustment of the true confocal output signal will nevertheless improve, overall, the resolution of the observation.
More preferably said method includes absorbing or otherwise excluding higher angle rays from said near confocal return light, whereby said near confocal return light comprises principally light from said overlap volume.
Preferably said method includes excluding higher angle rays from said near confocal return light by transmitting said near confocal return light through a region of said fiber provided with an outer cladding with a refractive index such that said higher angle rays are transmitted into said outer cladding while lower angle rays of said near confocal light are internally reflected and thereby retained in a glass inner cladding of said fiber.
Preferably said method includes absorbing light transmitted within said outer cladding.
In one embodiment, said fiber is a single moded optic fiber with a glass inner cladding and an outer cladding having a low refractive index such that modes of said near confocal return light in said glass cladding are normally internally reflected by said outer cladding, wherein said method includes cooling said outer cladding within a region of said fiber so that within said region said higher angle rays are transmitted into said outer cladding. More preferably said outer cladding comprises silicone rubber.
Preferably said outer cladding is surrounded at least partially within said region with an optically absorbing medium.
Preferably said cooling is by means of a Peltier effect cooler.
In one embodiment, said object is a data storage medium.
In a second broad aspect, the present invention provides an image sharpening apparatus for use in making a confocal microscopy or endoscopy observation, comprising:
a light condenser for collecting true confocal return light emanating from an observational field of an object, for focussing said true confocal return light into a core of a fiber wave-guide, for collecting near confocal return light from a volume partially overlapping said observational field and thereby defining an overlap volume, and for focussing said near confocal return light into said fiber wave-guide so as to be transmitted principally in a cladding of said fiber wave-guide;
detection means for detecting said true confocal return light and said near confocal return light, and to produce respectively a true confocal output signal and a near confocal output signal; and
signal processing means for adjusting said true confocal output signal on the basis of said near confocal output signal to substantially eliminate from said true confocal output signal a component due to said near confocal output signal;
whereby the effective volume of said observational field is reduced and the resolution of said observation is effectively increased.
Preferably said overlap volume is in the Z axis of said observational field.
Preferably said light condenser comprises a first light condenser and a second light condenser, wherein said first light condenser is arranged to collect and focus said true confocal return light and a second light condenser is arranged to collect and focus said near confocal return light.
More preferably said light condenser comprises a lens or a compound lens.
Preferably said detection means comprises a first detector and a second detector, wherein said first detector is arranged to detect said true confocal return light and said second detector is arranged to detect said near confocal return light.
In one embodiment, said signal processing means is operable to adjust said true confocal output signal by subtracting said near confocal output signal from said true confocal output signal.
Preferably said apparatus includes absorption means for extracting and absorbing higher angle rays from said near confocal return light, whereby said near confocal return light comprises principally light from said overlap volume.
Preferably said fiber has an glass inner cladding and a region provided with an outer cladding with a refractive index such that within said region higher angle rays of said near confocal return light are transmitted into said outer cladding while lower angle rays of said near confocal light are internally reflected and thereby retained in said glass cladding.
In one embodiment, said fiber is a single moded optic fiber with a glass inner cladding and an outer cladding having a low refractive index such that modes of said near confocal return light in said glass cladding are normally internally reflected by said outer cladding, wherein said apparatus includes means for increasing said refractive index of said outer cladding within a region of said fiber so that within said region said higher angle rays are transmitted into said outer cladding. More preferably said outer cladding comprises silicone rubber.
Preferably said outer cladding is surrounded at least partially within said region with an optically absorbing medium.
Preferably said means for increasing said refractive index of said outer cladding within a region comprises a cooling means, and more preferably a Peltier effect cooler.
Preferably said apparatus includes optical path varying means for varying the optical path of said true and near confocal return light to compensate for variations in said optical path due to changes in the depth of said observational field within said object, said optical path varying means having regions of greater and lesser optical path, whereby said optical path varying means can be located with a region of lesser optical path in said optical path when said observational field is deep within said object and with a region of greater optical path in said optical path when said observational field is less deep within said object.
Preferably said optical path varying means comprises an optical wedge.
In a third broad aspect, the present invention provides a data reading apparatus for reading data from a data storage medium, including the image sharpening apparatus described above.